The perioperative setting is considered a resource intensive section of the hospital. The logistics of sterilizing, sorting, building, and transporting surgical instruments is labor and capital intensive. In addition, infection due to improper sterilization is a safety hazard. Currently, processing of hundreds of types of surgical instruments with similar characteristics involves an extensive learning curve for hospital employees and diligent human performance.
Existing approaches to automating the sorting process are expensive and are limited in capability. One system currently utilized is designed to automate several key functions for the clean side of the sterile supply. With this system, a human operator first separates the surgical instruments from a pile, and places them on a conveyor belt one by one. Then a standard robotic arm fitted with a magnetic gripper picks up the single instrument, as isolated away from other instruments, from the belt. A machine vision system or a barcode scanner is used to identify the instrument and sort each instrument into a stack of similar instruments. Consequently, such solutions are limited when handling instrument manipulation in an unstructured environment where surgical instruments are cluttered in the container.
Picking individual items from an unordered pile in a container or in an unstructured environment with robotic equipment has proven overly burdensome and difficult. Picking surgical instruments is even more challenging. First, the surgical instruments often have similar characteristics, thus making it difficult for computer vision-based algorithms to recognize them from an unordered pile. Second, instruments are made of shiny metal. Optical effects such as specularities and inter-reflections pose problems for many computer-vision based pose estimation algorithms, including multi-view stereo, 3D laser scanning, active sensing (to acquire a depth map of the scene), and imaging with multiple light sources. One approach uses multi-flash cameras to obtain depth edge information and matches edge-based templates onto the input image for pose estimation. Edge-based template matching, however, has difficulty in distinguishing between very similar objects such as surgical instruments.
Typically, the current approaches have utilized each container having one object type. Most data-driven approaches to date use a multi-view approach to recognize objects in a scene and estimate their six degrees-of-freedom (DOF) poses. The approaches, however, rely on learned models for the objects. Since the surgical instruments have strong similarities, such a data-driven method is difficult to carry out. Without accurate six degrees-of-freedom (DOF) pose, many standard grippers have trouble executing the grip. Furthermore, when selecting the surgical instrument that is on top of the pile, typical appearance-based occlusion reasoning methods have trouble because objects which occlude others have a similar appearance as those being occluded.
It is therefore desirable to have a system as based on a vision-based bin-picking in unstructured environments as well as for use in current systems when selecting single objects. The invention would beneficially address current sterilization and safety hazards, while being capable of accurately singulating surgical instruments in a cluttered environment. The invention disclosure that follows addresses these challenges effectively.